Methods and systems for spatially localized image editing

ABSTRACT

This invention provides methods for spatially localized image editing. For example, an input image is divided into multiple bins in each dimension. For each bin, a histogram is computed, along with local image statistics such as mean, medium and cumulative histogram. Next, for each tile, a type of adjustment is determined and applied, including adjustment associated with Exposure, Brightness, Shadows, Highlights, Contrast, and Blackpoint. The adjustments are done for all tiles in the input image to render a small adjustment image. The small image is then interpolated, for example, using an edge-preserving interpolation, to get a full size adjustment image with adjustment curve for each pixel. Subsequently, per-pixel image adjustments can be performed across an entire input image to render a final adjusted image.

TECHNICAL FIELD

The disclosure generally relates to digital image editing.

BACKGROUND

When editing images, there are two “generic” types of editingadjustments. Global adjustments, such as tone curves, levels, andsaturation apply universally to the entire image. Local adjustments aretypically thought of to be “brushing” type adjustments where the usermanually “brushes” in adjustments to specific areas of the image thatthey are interested in. Global adjustments ignore differences betweendifferent local areas of an image and often result in an adjustmentimage that is either too dark in certain areas or too bright in certainareas.

Brush-type local adjustment often creates disconnection between brushedarea and its surrounding areas. Global adjustment is more efficient butoften leads to over exposure and under exposure. Global adjustment canalso create areas that are too saturated (e.g., in skin tones) or notsaturated enough (e.g., in colorful flowers or skies). In addition,applying brush-type local adjustment is manual, time consuming andtedious.

What is needed in the art are methods and systems that can balance thetwo type of approaches to create efficient and yet localized imageediting.

SUMMARY

In one aspect, disclosed herein are methods and systems for spatiallylocalized image editing. In some embodiments, the method comprise thesteps of rendering a downsampled adjustment image of an input image bydetermining an adjustment curve for each tile in a plurality of tiles ofthe input image, where each adjustment curve defines a relationship ofimage editing controls that is applied to pixels in the tile, where therelationship is derived from one or more image characteristics of eachtile in the plurality of tiles, the downsampled adjustment image havinga plurality of pixels, and where each pixel corresponds to theadjustment curve for a tile in the plurality of tiles; upsampling theadjustment image, by applying an edge preserving upsampling algorithm,to a full size adjustment image that is the same size as the inputimage, where each pixel in the full size adjustment image corresponds toa new adjustment curve; and performing pixel by pixel adjustment of eachpixel in the input image based on the new adjustment curve of acorresponding pixel in the full size adjustment image, thereby obtainingan edited version of the input image.

In one aspect, disclosed herein is an image upsampling process based ontwo different guide images.

In one aspect, disclosed herein is a non-transitory machine readablemedium storing an image-editing application executable by at least oneprocessor. Here, the image-editing application comprising sets ofinstructions for: dividing an input image into a plurality of tiles,where each tile comprises 10 or more pixels in either dimension;rendering a downsampled adjustment image by determining an adjustmentcurve for each tile in the plurality of tiles, where each adjustmentcurve defines a relationship of image editing controls that is appliedto pixels in the tile, where the relationship is derived from one ormore image characteristics of each tile in the plurality of tiles, thedownsampled adjustment image having a plurality of pixels, and whereeach pixel corresponds to the adjustment curve for a tile in theplurality of tiles; upsampling the adjustment image, by applying an edgepreserving upsampling algorithm, to a full size adjustment image that isthe same size as the input image, where each pixel in the full sizeadjustment image corresponds to a new adjustment curve; and performingpixel by pixel adjustment of each pixel in the input image based on thenew adjustment curve of a corresponding pixel in the full sizeadjustment image.

Details of one or more implementations are set forth in the accompanyingdrawings and the description below. Other features, aspects, andpotential advantages will be apparent from the description and drawings,and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an example system for performing spatiallylocalized editing of digital images.

FIG. 2 is an illustration of an example process of the spatiallylocalized image processing method.

FIG. 3 is flow diagram of an example process of the spatially localizedimage processing method.

FIG. 4 is flow diagram of an example process of the spatially localizedimage processing method.

FIG. 5 is a block diagram of an example system architecture implementingthe features and processes of FIGS. 1-4.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION Overview

FIG. 1 is a block diagram of an example system 100 for performingspatially localized image editing. For example, system 100 can performspatially localized adjustments to a digital image by dividing an inputimage into multiple bins, characterizing pixels in the individual binsto determine one or more adjustment curves specific for each bin, andcreating a downsampled adjustment image based on the bin-specificadjustment curves. The downsampled adjustment image is then upsampled tocreate new pixel-specific adjustment curves based on the bin-specificadjustment curve. In some embodiments, system 100 utilizes a number offunctional modules to achieve the various functionalities, includinginitial image processing module 110, intermediate image processingmodule 120, upsampling module 130 and final image processing module 140.

Unless otherwise specified, images disclosed herein refer to digitalimages. Also, unless otherwise specified, the terms “image processingmodule,” “processing module,” and “image module” can be usedinterchangeably.

In some embodiments, system 100 can include user device 102. Forexample, user device 102 can be a computing device such as a smartphone,a tablet computer, a laptop computer, or a wearable device (e.g., smartwatch, smart glasses, etc.).

In some embodiments, user device 102 includes a user input and outputmodule (I/O module) 104. For example, I/O module 104 can receive userinput to user device 102 and present output from user device 102 usingaudio, video, and/or haptic output mechanisms. For example, I/O module104 can receive touch input from a touch sensitive display of userdevice 102. In other embodiments, the user input and output module canbe coupled to other input devices, such as a keyboard, mouse, touchsensitive pad, etc., and receive user input from these devices.

In some embodiments, I/O module 104 can present audio, visual, and/orhaptic output to the user. For example, I/O module 104 can rendergraphical user interfaces (GUI) for performing spatially localized imageediting, as described herein. I/O module 104 can present GUIs thatenable or support the functionality of one or more of the imageprocessing modules described herein, including but not limited toinitial image processing module 110, intermediate image processingmodule 120, upsampling module 130 and final image processing module 140.In some embodiments, the user input and output module includes aplurality of menu commands, each corresponding to the functionality ofone or more of the image processing modules. In some embodiments, I/Omodule 104 can allow a user to enter voice commands for performingspatially localized image editing, as described herein. Additionally,for example, I/O module 104 can allow a user to enter commands bytapping a touch screen.

In some embodiments, user device 102 can include image database 106. Forexample, image data base 106 includes images that are taken and saved onuser device 102. In some embodiments, a user can store images at variousstages of processing in image database 106. The functionality isachieved by connecting different processing modules (e.g., modules 110,120, 130 and 140) to a centralized image database.

In some embodiments, user device 102 can include initial imageprocessing module 110. For example, a user can select (e.g., via userinput and output module 104) an image (e.g., input image 210 in FIG. 2)stored in image database 106 for editing. In some implementations,initial image processing module 110 can divide input image 210 into aplurality of tiles, grids, or bins. As disclosed herein, the terms“bin,” “grid” and “tile” are used interchangeably.

In some embodiments, a user can choose from pre-defined settings; e.g.,from a pull-down or pop-up menu to select a specific bin setting amongmultiple settings predefined based on commonly available sizes ofdigital images. For example, an input image of 680 pixel×850 pixel canbe divided into a 34×34 grid map, where each grid includes an array of20 pixel×25 pixel (a total of 500 pixels). Alternatively, the same inputimage can be divided into a 20×25 grid map, where each grid include anarray of 34 pixel×34 pixel (a total of 1156 pixels). It will beunderstood that the size of the grid map (and the number of bins) can bedetermined based on the size of an input image.

Initial image processing module 110 characterizes image data from inputimage 210 at individual bin level. For example, each bin includes anumber of pixels (e.g., m×n pixels) and these pixels are furtheranalyzed by initial image module 110. For example, the user may wish toedit an image to change exposure level of selected areas of input image210. Using spatially localized editing, a user can adjust the exposurelevel of each bin individually, avoiding over-exposure or under-exposurethat is often associated with a global adjustment operation. Initialimage processing module 110 can identify exposure level of all pixels ineach bin and compiles a profile (e.g., a histogram). An image histogramis a type of histogram that acts as a graphical representation of thetonal or color distribution in a digital image.

In some embodiments, initial image processing module 110 can alsocompile data concerning additional characteristics such as colordistribution for all color channels; e.g., red green and blue (RGB)channels or cyan magenta yellow and black (CMYK) channels. In someembodiments, initial image processing module 110 can covert input image210 into a grey image to calculate a grey scale histogram. Afterhistograms are generated for each bin, initial image processing module110 can further analyze the histograms to derive a bin-specificadjustment curve. The bin-specific adjustment curve defines therelationships for editing all pixels in the particular bin. Additionaldetails concerning adjustment curves can be found in description of FIG.2. As disclosed herein, the terms “adjustment curve,” “adjustmentfunction,” “adjustment algorithm,” and “adjustment relationship” can beused interchangeably unless otherwise specified.

In some embodiments, the user can choose to save data from initial imageprocessing module 110 in image data base 106. Alternatively, the usercan send the initially processed images to intermediate image processingmodule 120 for further processing. For example, a user can select one ormore commands (e.g., “save image data” and “continue to process image”)through via I/O module 104 to execute one or both options.

In some embodiments, user device 102 can include intermediate imageprocessing module 120. Intermediate image processing module 120organizes adjustment curves of each individual bin to generate adownsampled image (e.g., adjustment image 230). For example, module 120can render adjustment image 230 based on input image 210. Each “pixel”in the downsampled image is associated with one or more adjustmentcurves of a corresponding bin in original input image 210. For example,bin B_((x,y)) in input image 210 corresponds to pixel P_((x,y)) inadjustment image 230. In the adjustment image 230, each pixel isassociated with a corresponding adjustment curve. In some embodiments,intermediate image processing module 120 creates an adjustment image 230for each of exposure, brightness, shadows, highlights, contrast, andblackpoint. In some embodiments, adjustment image 230 corresponds to twoor more of the image characteristics in any possible combination. Asdisclosed herein, the terms downsampled adjustment image and“downsampled image” are be used interchangeably unless otherwisespecified.

In some embodiments, a user can choose to save the adjusted (anddownsampled) images in image database 106. Alternatively, a user canchoose to continue to process the downsampled images using upsamplingmodule 130. For example, a user can select one or more commands (e.g.,“save image data” and “continue to process image”) through user inputand output module 104 to execute one or both options. In someembodiments, the functionalities of the initial image module 110 andintermediate image processing module 120 are combined in one imagemodule.

In some embodiments, user device 102 can include upsampling module 130.In some embodiments, upsampling module 130 can generate an upsampledimage based on the adjusted and downsampled images (e.g., adjustmentimage 230 in FIG. 2). For example, upsampling module 130 can receivedownsampled images from intermediate module 120, or database 106, andgenerate upsampled image 240 based on the downsampled images. Forexample, module 130 can generate the upsampled image using an upsamplingalgorithm, such as nearest-neighbor interpolation, bilinearinterpolation, bicubic interpolation, Lanczos resampling, Fourier-basedinterpolation, edge-directed interpolation algorithms, and etc.Preferably, edge-directed interpolation algorithms are used becausethese algorithms aim to preserve edges in the image after scaling,unlike other algorithms which can produce staircase artifacts arounddiagonal lines or curves. Examples of edge preserving algorithms includebut are not limited to New Edge-Directed Interpolation (NEDI),Edge-Guided Image Interpolation (EGGI), Iterative Curvature-BasedInterpolation (ICBI), and Directional Cubic Convolution Interpolation(DCCI).

In some embodiments, a fast edge preserving upsampling algorithm isapplied during the upsampling process. Additional details concerning themethod can be found in application Ser. No. ______ (Attorney Docket No.337722-318500/P30306), entitled “FAST AND EDGE-PRESERVING UP SAMPLING OFIMAGES” and filed concurrently herewith, which is hereby incorporated byreference in its entirety.

During upsampling, one bin-specific adjustment curve is converted tomultiple pixel-specific adjustment curves; e.g., m×n pixel-specificadjustment curves, depending on the number of pixels included in theparticular bin. Details of the conversion from one bin-specificadjustment curve to multiple pixel-specific adjustment curves can befound in the description of upsampled image 240 of FIG. 2 and thedescription of steps 308 and 310 of FIG. 3.

In some embodiments, the upsampled image is of the same size as theinput image. In some embodiments, the upsampled image is of a differentsize from the input image.

In some embodiments, a user can save the upsampled images in imagedatabase 106. Alternatively, the user can choose to further process theupsampled images using final image processing module 140. For example, auser can select one or more commands (e.g., “save image data” and“continue to process image”) through user I/O module 104 to execute oneor both options.

Functionalities of different modules can be performed separately byindividual modules. Multiple functionalities can also be performed inone functional module. In some embodiments, the functionalities ofinitial image module 110 and intermediate image processing module 120are combined in one image module. In some embodiments, thefunctionalities of initial image module 110, intermediate imageprocessing module 120, and upsampling module 130 are combined in oneimage module. In some embodiments, the functionalities of initial imagemodule 110, intermediate image processing module 120, upsampling module130 and final image processing module 140 are combined in one imagemodule.

FIG. 2 is an illustration 200 of an example process of a spatiallylocalized image processing method. As shown in illustration 200, inputimage 210 is first downsampled to adjustment image 230 and thenupsampled to upsampled image 240. Information from upsampled 240 arecombined with input image 210 to render a final image 250.

As an initial step, input image 210 is subdivided into a plurality ofbins; e.g., by initial image processing module 110. As illustrated inFIG. 2, input image of 12×20 is divided into a 4×5 grid map. Each bin inthe grid map can be labeled as B_((x,y)), where x and y are twodimensional axes used to specify the location of the particular bin.Here, the value for x is an integer from 1 to 5 and the value for y isan integer from 1 to 4. For example, the highlighted grid in input image210 can be designated as is B_((1,3)). In the simple illustration, thegrid includes 12 pixels arranged in a 3 pixel×4 pixel array.

In some embodiments, initial image processing module 110 can analyzedigital image characteristics of each grid to compile image histograms.A histogram is a bar graph. In a tone histogram, the x-axis representsthe tonal scale (black at the left and white at the right), and y-axisrepresents the number of pixels in an image in a certain area of thetonal scale. For example, the graph of a luminance histogram shows thenumber of pixels for each brightness level (from black to white), andwhen there are more pixels of a particular brightness level, the peak atthe certain luminance level is higher.

A color histogram is a representation of the distribution of colors inan image. For digital images, a color histogram represents the number ofpixels that have colors in each of a fixed list of color ranges, thatspan the image's color space, the set of all possible colors. In someembodiments, color histograms can be established for color distributionfor all color channels; e.g., red green and blue (RGB) channels or cyanmagenta yellow and black (CMYK) channels. For example, a red histogramcan show how the intensity of the color red is distributed throughoutthe bin. In some embodiments, initial image processing module 110 cancovert input image 210 into a grey image to calculate a grey scalehistogram. For example, it is possible to calculate a saturationhistogram to determine how much saturation/desaturation to be applied toa particular bin. In some embodiments, the level ofsaturation/desaturation can be applied to multiple bins. In someembodiments, it is possible to examine the standard deviation of the binto determine how much sharpening to be applied to the bin. In someembodiments, the level of sharpening can be applied to multiple bins.

After histograms are generated for each bin, initial image processingmodule 110 can further analyze the histograms to derive a bin-specificadjustment curve. The bin-specific adjustment curve definesrelationships for editing the pixels in the particular bin. In someembodiments, the adjustment curve is a cumulative histogram curve. Forexample, a cumulative histogram is a mapping that counts the cumulativenumber of observations (such as brightness level) in all of the pixelsin a particular bin. Cumulative histogram can be used to identify areaswithin an image that is over-exposed or under-exposed. In someembodiment, initial image processing module 110 can apply a simplelinear function to tone down a bin with over-exposed pixels or enhance abin with under-exposed pixels. In some embodiments, the adjustment curveis a standard tone curve. For example, if a user wants to adjust theexposure level of pixels in the particular bin, an exposure adjustmentcurve can be automatically calculated; e.g., based on a medium exposurelevel of each bin. Additionally, a user can input a desired exposurelevel via I/O module 104 and initial image processing module 110 cangenerate an adjustment curve based on the input value. As disclosedherein, initial image processing module 110 defines a bin-specificadjustment curve for each bin. As such, in the example illustrated inFIG. 2, twenty bin-specific adjustment curves will be defined for inputimage 210.

Immediate image processing module 120 then applies each of the twentybin-specific adjustment curves to a corresponding bin such that pixelswithin the particular bin are adjusted. After the adjustment, adjustmentimage 230 can be created as an adjusted and downsampled version of inputimage 210. For example, after the 12 pixels in bin B_((1,3)) areadjusted, the adjusted pixels can be downsampled to one pixel; forexample, the immediate image processing modules can take the intensityvalues for each color channel from 12 pixels and calculate a coloraverage for each channel. As such, in some embodiments, 12 pixels inB_((1,3)) of input image 210 is converted to pixel P_((1,3)) inadjustment image 230. Immediate image processing module 120 organizeslocal adjustment for each bin in input image 210 to convert input image210, which is 12 pixel×20 pixel, to adjustment image 230, which is 4pixel×5 pixel. Additionally, any known algorithm for downsampling can beused for creating adjustment image 230. Example algorithms include butare not limited to nearest-neighbor interpolation, bilinearinterpolation, bicubic interpolation, Lanczos resampling, Fourier-basedinterpolation, edge-directed interpolation algorithms, and etc.Preferably, edge-directed interpolation algorithms are used becausethese algorithms aim to preserve edges in the image after scaling,unlike other algorithms which can produce staircase artifacts arounddiagonal lines or curves. Example edge preserving algorithms include butare not limited to New Edge-Directed Interpolation (NEDI), Edge-GuidedImage Interpolation (EGGI), Iterative Curvature-Based Interpolation(ICBI), and Directional Cubic Convolution Interpolation (DCCI).

In some embodiments, a user can choose to save adjustment image 230 inimage databased 106. Alternatively, a user can choose to continueprocessing adjustment image 230 at upsampling module 130. For example, auser can select one or more commands (e.g., “save image data” and“continue to process image”) through user I/O module 104 to execute oneor both options.

Next, upsampling module 130 takes adjustment image 230 as input andapplies an upsampling algorithm to create upsampled image 240.Preferably, the upsampling algorithm includes edge-directedinterpolation algorithms to preserve edges in the image after scaling,unlike other algorithms which can produce staircase artifacts arounddiagonal lines or curves. Example edge preserving algorithms include butare not limited to New Edge-Directed Interpolation (NEDI), Edge-GuidedImage Interpolation (EGGI), Iterative Curvature-Based Interpolation(ICBI), and Directional Cubic Convolution Interpolation (DCCI).

In some embodiments, upsampling module 130 applies a fast edgepreserving upsampling algorithm during the upsampling process.Additional details concerning the method can be found in applicationSer. No. ______ (Attorney Docket No. 337722-318500/P30306), entitled“FAST AND EDGE-PRESERVING UPSAMPLING OF IMAGES” and filed concurrentlyherewith, which is hereby incorporated by reference in its entirety.

After upsampling, adjustment image 230 (a 5×4 grid map) is turned intoupsampled image 240 (a 20×16 grid map). In upsampled image 240, x′ andy′ are the two dimensional axes that can be used to specify the locationof pixels within the image. Here, the value for x′ is an integer from 1to 20 and the value for y′ is an integer from 1 to 16. As such, pixelP_((1,3)) in adjustment image 230 itself is transformed back to an arrayof 3 pixel×4 pixel, as indicated by the patterned area in upsampledimage 240:

-   -   {P_((1,7)), P_((2,7)), P_((3,7)), P_((4,7))};    -   {P_((1,8)), P_((2,8)), P_((3,8)), P_((4,8))}; and    -   {P_((1,9)), P_((2,9)), P_((3,9)), P_((4,9))}.

As noted above, pixel P_((1,3)) is associated with an adjustment curve.Based on the values of the pixels in the upsampled array, upsamplingmodule 130 can calculate a new pixel-specific adjustment curve for eachpixel: each pixel in the pixel array is associated with its ownadjustment function.

In preferred embodiments, upsampled image 240 is of the same size asinput image 210. In some embodiments, upsampled image 240 may be largeror smaller than input image 210.

In some embodiments, a user can choose to save upsampled image 240 inimage databased 106. Alternatively, a user can choose to continueprocessing upsampled image 240 at final image processing module 140. Forexample, a user can select one or more commands (e.g., “save image data”and “continue to process image”) through user I/O module 104 to executeone or both options.

Final image processing module 140 applies a pixel-specific adjustmentcurve from a pixel in upsampled image 240 to the corresponding pixel ininput image 210. For example, final image processing module 140 appliesthe new pixel-specific adjustment curve of pixel P_((3,8)) in upsampledimage 240 to pixel P_((3,8)) in input image 210. Final image processingmodule 140 applies pixel-by-pixel adjustment to each pixel in inputimage 210 to render final image 250. As disclosed herein, the terms“final image,” “final adjusted image,” and “result image” are usedinterchangeably unless otherwise specified.

In some embodiments, each of the types of images (e.g., input image 210,adjustment image, upsampled image 240 and final image 250) is handled bya dedicated functional modules (e.g., initial image processing module110, intermediate image processing module 120, upsampling module 130 andfinal image processing module 140). In some embodiments, thefunctionalities of initial image module 110 and intermediate imageprocessing module 120 are combined in one image module. In someembodiments, the functionalities of initial image module 110,intermediate image processing module 120, and upsampling module 130 arecombined in one image module. In some embodiments, the functionalitiesof initial image module 110, intermediate image processing module 120,upsampling module 130 and final image processing module 140 are combinedin one image module.

Example Processes

FIG. 3 is flow diagram of an example process 300 of spatially localizedimage processing method. For example process 300 can be performed byuser device 102 to adjust or edit characteristics of an image. The flowdiagram illustrates how an input image is divided into multiple bins anddownsampled to an adjustment image corresponding to bin-specific localadjustment. The adjustment image is then upsampled to an upsampled imageto create pixel-specific adjustment functions, which are then applied,pixel-by-pixel, to the input image to create a final adjusted image.

At step 302, user device 102 can divide an input image into multiplebins or tiles. In some embodiments, the input image is divided into am×n grid, wherein m and n are integers that are equal or greater than 5.In some embodiments, the number of grids corresponds to the number ofpixels. For example, m can include 10 or more, 50 or more, or 500 ormore pixels. Similarly, n can include 10 or more, 50 or smaller, or 500or more pixels. In some embodiments, m and n have the same value. Insome embodiments, m and n have different values. As noted above, thesize of a grid map can be determined based on the dimensions of commonlyavailable digital image format. For example, an image 1024 pixel×768pixel can be divided into a 32×32 grid map (m=32 and n=24), while a 800pixel×600 pixel image can be divided into a 20×20 grid map (m=40 andn=30). In some embodiments, an image of 2028×1536 can divided in a 32×24grid map (m=64 and n=64).

At step 304, image characteristics of the pixels within each bin aredetermined; e.g., by using initial image processing module 110. Imagecharacteristics are associated with image features such as color, tone,exposure, contrast, shadows, brightness, blackpoint and etc. Sampleimage characteristics include but are not limited to a color histogram,a grey histogram, a cumulative histogram, statistics, or a user input.

In a tone histogram, the x-axis represents the tonal scale (black at theleft and white at the right), and y-axis represents the number of pixelsin an image in a certain area of the tonal scale. For example, the graphof a luminance histogram shows the number of pixels for each brightnesslevel (from black to white), and when there are more pixels of aparticular brightness level, the peak at the certain luminance level ishigher.

A color histogram is a representation of the distribution of colors inan image. For digital images, a color histogram represents the number ofpixels that have colors in each of a fixed list of color ranges, thatspan the image's color space, the set of all possible colors. In someembodiments, color histograms can be established for color distributionfor all color channels; e.g., red green and blue (RGB) channels or cyanmagenta yellow and black (CMYK) channels. For example, a red histogramcan show how the intensity of the color red is distributed throughoutthe bin. In some embodiments, initial image processing module 110 cancovert input image 210 into a grey image to calculate a grey scalehistogram.

After histograms are generated for each bin, initial image processingmodule 110 can further analyze the histograms to derive a bin-specificadjustment curve. The bin-specific adjustment curve definesrelationships for editing the pixels in the particular bin. In someembodiments, the bin-specific adjustment curve is a cumulative histogramcurve. For example, a cumulative histogram is a mapping that counts thecumulative number of observations (such as brightness level) in all ofthe pixels in a particular bin. Cumulative histogram can be used toidentify areas within an image that is over-exposed or under-exposed. Insome embodiments, a plurality of histograms are calculated for each bin.In some embodiments, a cumulative histogram is calculated for each bin.In some embodiments, colored images are analyzed. In some embodiments,colored images are converted to grey scale before characterization.

At step 306, intermediate image processing module 120 determines therelationships of image editing controls for pixels in each bin. Forexample, after histograms are generated for each bin, initial imageprocessing module 110 can further analyze the histograms to derive abin-specific adjustment curve. The bin-specific adjustment curve definesrelationships for editing the pixels in the particular bin. In someembodiments, the adjustment curve is a cumulative histogram curve. Forexample, a cumulative histogram is a mapping that counts the cumulativenumber of observations (such as brightness level) in all of the pixelsin a particular bin. In some embodiment, intermediate image processingmodule 120 can apply a simple linear function to tone down a bin withover-exposed pixels or enhance a bin with under-exposed pixels. In someembodiments, the bin-specific adjustment curve is a standard tone curve.For example, if a user wants to adjust the exposure level of pixels inthe particular bin, an exposure adjustment curve can be automaticallycalculated; e.g., based on a medium exposure level of each bin.Additionally, a user can input a desired exposure level via I/O module104 and intermediate image processing module 120 can generate anadjustment curve based on the input value. As disclosed herein, initialimage processing module 110 defines a bin-specific adjustment curve foreach bin. As such, in the example illustrated in FIG. 2, twentybin-specific adjustment curves will be defined for input image 210.

In some embodiments, relationships between image editing controls aredefined in one or more bin-specific adjustment curves. In someembodiments, an image editing control is defined based on one or moreimage characteristics. Example image editing controls include but arenot limited to exposure controls, brightness controls, shadows controls,highlights controls, contrast controls, blackpoint controls. In someembodiments, a bin-specific adjustment curve defines the relationshipfor one image characteristic. For example, as noted above, a simplelinear function may be used to tone down a bin with over-exposed pixelsor enhance a bin with under-exposed pixel. In some embodiments, abin-specific adjustment curve defines the relationship for two or moreimage characteristics. For example, exposure level, brightness level andcontrast level are related to each and can be adjusted using the samebin-specific adjustment function.

In some embodiments, the bin-specific adjustment curve is a polynomialfunction; for example, a second order polynomial function:

y=ax ²+(1−a)x,  (1)

where x represents an image characteristic prior to adjustment such asthe intensity value of a color, y represents the adjusted value, and ais the adjustment value. The following example illustrates howadjustment value in a second order polynomial function is determined.Here, a bin in the original input image included a RGB color channels aswell as brightness. As disclosed herein, a can be associated with one ormore bin-specific adjustment curves. In some embodiments, a representswhat is stored in downsampled adjustment image 230. In some embodiments,after upsampling, a is associated with one or more pixel-specificadjustment curves for pixels in the upsampled adjustment image. In someembodiments, a represents what is stored in upsampled adjustment image240.

For the example second order polynomial function: y=ax²+(1−a)x,luminance value (Y) can be used to compute the value of a:

adjusted value=a*Y _((x,y)) *Y _((x,y))+(1−a)*Y _((x,y))  (2)

Here, Y_((x,y)) is the luminance value for a pixel P_((x,y)). A seriesof equations can be obtained for all pixels in the bin. When the desiredvalue is set as the median luminance value (e.g., 0.435), the equationseries can be used to solve a such that half of the adjusted values areabove the medium luminance value and half of the adjusted values arebelow the medium luminance value.

At step 308, intermediate processing module 120 can apply a bin-specificadjustment curve previously determined at step 306 to pixels in theselected bin. Intermediate processing module 120 applies image editingto each bin based on a corresponding bin-specific adjustment curvedetermined for the particular bin. In some embodiments, a standard tonecurve is applied. In some embodiments, intermediate processing module120 applies a bin-specific adjustment curve that modifies or optimizesone or more image characteristics. In some embodiment, intermediateprocessing module 120 applies a bin-specific adjustment curve thatmodifies or optimizes one or more image control. At this step,intermediate processing module 120 applies bin-specific adjustment toeach bin instead of a global adjustment that is applied to the entireimage. In particular, the same adjustment is applied to pixels withinthe same bin. Essentially, the bin-based image editing is a localadjustment: the total number of bin-specific adjustment curvecorresponds to the number of bins. After the pixels of each bin isadjusted, a downsampled adjustment image is created where the number ofpixels in the adjustment image corresponds to the number of adjustmentcurves that is used to edit the bins of the input image. Intermediateprocessing module 120 can apply any known algorithm for resizing animage (e.g., including downsizing and upsizing). Example algorithmsinclude but are not limited to nearest-neighbor interpolation, bilinearinterpolation, bicubic interpolation, Lanczos resampling, Fourier-basedinterpolation, edge-directed interpolation algorithms, and etc.Preferably, edge-directed interpolation algorithms are used becausethese algorithms aim to preserve edges in the image after scaling,unlike other algorithms which can produce staircase artifacts arounddiagonal lines or curves. Example edge preserving algorithms include butare not limited to New Edge-Directed Interpolation (NEDI), Edge-GuidedImage Interpolation (EGGI), Iterative Curvature-Based Interpolation(ICBI), and Directional Cubic Convolution Interpolation (DCCI).

Once the a value is determined, the same equation can be used tocalculate color values of each pixel in the bin, based on the followingequations:

newRed=a*origRed*origRed+(1−a)*origRed  (3)

newGreen=a*origGreen*origGreen+(1−a)*origGreen  (4)

newBlue=a*origBlue*origBlue+(1−a)*origBlue  (5)

A user-controlled slider then could be used to adjust how close towards“a” the adjustment goes based on the following

a=(sliderValue)*a  (6)

So when the slider is at 0, nothing happens (the above equations (3)-(5)solve to origRed, origGreen, origBlue), and when the slider is at 1,full “corrections” are obtained so that each area would have the sameaverage luminance.

In some embodiments, intermediate image processing module 120 appliesone or more image adjustment algorithms at step 306.

In a simple example, as illustrated in connection with FIG. 2,intermediate processing module 120 can calculate an average for theadjusted pixels in a particular bin.

At step 310, upsampling module 130 upsamples the downsampled adjustmentimage to create a full size upsampled image. In some embodiments, anedge preserving upsampling algorithm (such as one of those disclosedabove) is applied during the upsampling process. After upsampling, apixel-specific new adjustment curve is created for each pixel in theupsampled image, as illustrated in FIG. 2. For example, afterupsampling, pixel P_((1,3)) in adjustment image 230 is expanded to anarray of 3 pixel×4 pixel, as indicated by the patterned area inupsampled image 240:

-   -   {P_((1,7)), P_((2,7)), P_((3,7)), P_((4,7))},    -   {P_((1,8)), P_((2,8)), P_((3,8)), P_((4,8))}; and    -   {P_((1,9)), P_((2,9)), P_((3,9)), P_((4,9))}.

Based on the adjusted values of pixels in the expanded pixel array,upsampling module 130 can calculate new pixel-specific adjustment curvesbased on the original bin-specific adjustment curve. For example, incomparison to the pixel P_((1,3)) in adjustment image 230, a colorintensity of upsampled pixel P_((3,8)) in upsampled image 240 may havebeen slightly adjusted during upsampling. A new adjustment curve forpixel P_((3,8)) can be calculated based on the bin-specific adjustmentcurve and the differences between pixel P_((1,3)) in adjustment image230 and pixel P_((3,8)) in upsampled image 240.

Similar to bin-specific adjustment curves, pixel-specific adjustmentcurves controls include but are not limited to exposure controls,brightness controls, shadows controls, highlights controls, contrastcontrols, blackpoint controls. In some embodiments, an adjustment curvedefines the relationship of two, three, four, five, or six image editingcontrols.

Example adjustment curve includes but is not limited to a polynomialfunction, a linear function, a gamma function, an exponential function,a linear equation, a sigmoid function, and combinations thereof.

In some embodiments, the adjustment curve is a second order polynomialbased tone curve: y=ax²+(1−a)x, x represents an input data value such asthe intensity of a particular color in a particular pixel, y representsan output result, and a is a value specific to all pixels in each tile,where x and y correspond to pixel values of one or more selected fromthe group consisting of exposure, brightness, shadows, highlights,contrast, blackpoint and combinations thereof.

A second order polynomial function is advantageous for a number ofreasons. First, it is similar to common tone-curves and can both lighten(when a is less than 0) or darken (when a is greater than 0) in apleasing manner. Second, a second order polynomial is also quick tocompute on the GPU, when compared to other tone curves. And lastly,solving for a in a second order polynomial function is a simpleclosed-form set of linear equations.

Different functions can be used under different circumstances. Forexample, a “gamma” or exponential function can be used, however such afunction may be too strong in lifting the shadows, causing images tolook washed out. Additionally, in some embodiments, solving simplelinear equations work well in the shadows, but may make the highlightsgray and murky. A more complicated “sigmoid” function may be applicableas well, such as a hyperbolic-tangent or a cumulative normal; however,those are much more computationally expensive to apply.

In some embodiments, a simple linear gain is used for only adjusting“shadows.” In some embodiments, a polynomial function is used to performsix adjustments at once, including exposure, brightness, shadows,highlights, contrast, and blackpoint.

In some embodiments, a slider can be used to adjust the values of one ormore parameters.

At step 312, new pixel-specific adjustment curves obtained from theprevious step is applied to input image 210 for pixel-by-pixel imageediting. As discussed above in connection with step 310, each upsampledpixel is associated with a new pixel-specific adjustment curve. Inpreferred embodiments, full size upsampled image 240 is of the same sizeas input image 210 such that each pixel in the upsampled image has acorresponding pixel in input image 210. For example, the new adjustmentcurve for pixel P_((3,8)) in upsampled image 240 can be applied tocorresponding pixel P_((3,8)) in input image 210 to generate a finaledited pixel P_((3,8)).

At step 314, final image processing module 140 applies pixel-by-pixelediting to each pixel in input image 210 to generate final edited pixelscorresponding to all pixels in input image 210. The result is finalimage 250.

FIG. 4 illustrates an example image processing process 400, showing apreferred embodiment of spatially localized image editing method. Forexample, process 400 can be performed by user device 102, describedabove.

At step 402, an input digital image is divided into a plurality of binsor tiles. Each bin or tile is a two dimensional area that comprisemultiple pixels in either dimension. For example, either dimension caninclude 10 or more, 50 or more, or 100 or more pixels.

At step 404, one or more image characteristics are determined for eachbin of the input image, as discussed in connection with initial imageprocessing module 110 in FIG. 1, step 220 in FIG. 2 and step 304 in FIG.3.

At step 406, an adjustment curve is determined for each bin to create adownsampled adjustment image of the input image. The downsampledadjustment image comprises a plurality of pixels, each corresponding toone or more adjustment curves for a bin of the plurality of bins. Eachadjustment curve defines a relationship of image editing controls thatis applied to pixels in the tile. The relationship is derived from oneor more image characteristics of each tile in the plurality of tiles.Additional details concerning step 406 can be found in the descriptionof initial image processing module 110 and intermediate processingmodule in FIG. 1, the description of step 220 and adjustment image 230in FIG. 2, and the description of steps 306 and 308 in FIG. 3.

At step 408, the downsampled adjustment image is upsampled to a fullsize adjustment image that is of the same size as the original inputimage. Here, an edge preserving upsampling algorithm is applied. Eachpixel in the full size adjustment image has a corresponding newadjustment curve. Additional details concerning step 408 can be found inthe description of upsampling module 130 in FIG. 1, the description ofupsampled image 240 in FIG. 2, and the description of step 310 in FIG.3.

At step 410, each new adjustment curve from the full size adjustmentimage is applied to a corresponding pixel in the original input digitalimage such that pixel by pixel adjustment or image editing is performedfor each pixel in the input image. At step 412, after the pixel by pixeladjustment or image editing, a new or final adjusted image is created.Additional details concerning steps 410 and 412 can be found in thedescription of final image processing module 140 in FIG. 1, thedescription of final image 250 in FIG. 2, and the description of steps312 and 314 in FIG. 3.

Spatially Localized Image Editing

According to the method and system disclosed herein, an input image canbe divide into multiple bins in each dimension, for example, to create a32×32 tile or grid map. Here, the 32×32 tile or grid map is usedthroughout as an example to illustrate the spatially localized imaginemethod and system as disclosed herein. It will be understood that animage can be divided in a grid map of any suitable size. For each imagetile in a tile map, a histogram of the local image area is computed,along with local image statistics including mean, medium and cumulativehistogram. Next, for each image tile, a type of adjustment is determinedand applied to the area. The type of adjustment can include, forexample, exposure, brightness, shadows, highlights, contrast, andblackpoint. The adjustments are done for all tiles in the input image,and then are interpolated across the image, for example, using anedge-preserving interpolation, to get per-pixel image adjustments acrossan entire input image.

Specifically, an input image is broken down into discrete “tiles”,usually 32×32 tiles across the image size. For each of these imagetiles, the histogram of the local image area is computed. Additionally,local image statistics such as mean, median, and cumulative histogramare computed. This gives us an approximation of how well “exposed” eacharea of the image is. From there, the type of adjustment that can beapplied to each specific region is computed. For example, a photoediting application can provide a slider control for adjusting the“light” of a selected image. The “light” adjustment can globally adjustmultiple characteristics of the image, such as exposure, brightness,shadows, highlights, contrast, and blackpoint simultaneously, based uponthe global statistics of the image; see, for example, U.S. Pat. No.9,323,996, which is hereby incorporated by reference in its entirety.Although the previously disclosed principals still apply, methods andsystems disclosed herein can calculate adjustments that can be appliedindividually to each of the 32×32 smaller image tiles, thus allowinglocal adjustments of different areas of an image instead of globaladjustment of the entire image as a whole. This type of spatiallylocalized adjustment could also be extended to any other “automatic”adjustment, such as automatic curves, levels, etc.

Once the smaller 32×32 grid of adjustments are obtained, a way is neededto figure out how to adjust the full-size image, based upon those localadjustments. Here, the 32×32 array of adjustments is treated as a tinyimage. The image is then upsampled to the full-size of the originalimage, using a typical image upscaling such as bilinear or cubicinterpolation, but that technique has drawbacks. Consider an area of theimage that transitions from dark (i.e., underexposed) to bright (i.e.,overexposed) at a sharp edge, say a person standing in front of a windowor backlit by the sky. If the image is simply interpolated across thatdark-to-light edge, a halo would appear, because the under-exposedperson would first get too dark (e.g., as the window exposure adjustmentbleeds into the face) and then get too light as the correct adjustmentof the face comes into play. An example of upscaling a small image isshown below.

To avoid these “halo” artifacts, the original image is as a “guide” toupsample the small adjustment thumbnail to the full size of the image.Common techniques for this type of up sampling are the Joint BilateralUpsample filter, or the Guided Filter. Either of these techniques can beimplemented with an edge preservation.

Once the full-sized adjustment image is obtained, it could be used as alook-up to get a per-pixel image adjustment. For instance, if theadjustment image is an exposure adjustment, both the original image andthe adjustment were passed into the our filter. For each pixel in theoriginal, the same pixel of the adjustment image would be examined todetermine how to adjust the exposure for that given pixel.

Graphical User Interfaces and Image Editing Control

This disclosure further includes various Graphical User Interfaces(GUIs) for implementing various features, processes or workflows. Forexample, the GUIs can be implemented through I/O module 104. These GUIscan be presented on a variety of electronic devices including but notlimited to laptop computers, desktop computers, computer terminals,television systems, tablet computers, e-book readers and smart phones.One or more of these electronic devices can include a touch-sensitivesurface. The touch-sensitive surface can process multiple simultaneouspoints of input, including processing data related to the pressure,degree or position of each point of input. Such processing canfacilitate gestures with multiple fingers, including pinching andswiping.

When the disclosure refers to “select” or “selecting” user interfaceelements in a GUI, these terms are understood to include clicking or“hovering” with a mouse or other input device over a user interfaceelement, or touching, tapping or gesturing with one or more fingers orstylus on a user interface element. User interface elements can bevirtual buttons, menus, selectors, switches, sliders, scrubbers, knobs,thumbnails, links, icons, radio buttons, checkboxes and any othermechanism for receiving input from, or providing feedback to a user.

In some embodiments, the GUIs include multiple image editing controls.For example, each image editing control can be used to adjust the valuesfor one or more image parameters. In some embodiments, a parameter asdisclosed herein includes image characteristics such as exposure,brightness, shadows, highlights, contrast, blackpoint and combinationsthereof. In some embodiments, a parameter as disclosed herein includesconstants or variables in an adjustment curve that helps to define therelationships between image characteristics such as exposure,brightness, shadows, highlights, contrast, and blackpoint.

In some embodiments, one or more menu items can be employed as imageediting controls. For example, the menu items can be part of an imageadjustment menu of the GUIs. Each menu item can be used to set thevalues of one or more image parameters. For example, a menu item can bea slider bar. A user can change the position of the slider bar to definea desirable value for one or more parameters in an adjustment curve. Forexample, a user can enter a desired brightness level for a particularbin and intermediate image processing module 120 can determine anadjustment curve based on the desired brightness level. In someembodiments, a menu item (i.e., an image editing control) can adjust thevalues of two different parameters, three parameters, four parameters,five parameters, or six or more parameters. For example, setting themedium exposure level may also change the values for brightness level,contrast level, shadow level and etc.

Example System Architecture

FIG. 5 is a block diagram of an example computing device 500 that canimplement the features and processes of FIGS. 1-4. The computing device500 can include a memory interface 502, one or more data processors,image processors and/or central processing units 504, and a peripheralsinterface 506. The memory interface 502, the one or more processors 504and/or the peripherals interface 506 can be separate components or canbe integrated in one or more integrated circuits. The various componentsin the computing device 500 can be coupled by one or more communicationbuses or signal lines.

Sensors, devices, and subsystems can be coupled to the peripheralsinterface 506 to facilitate multiple functionalities. For example, amotion sensor 510, a light sensor 512, and a proximity sensor 514 can becoupled to the peripherals interface 506 to facilitate orientation,lighting, and proximity functions. Other sensors 516 can also beconnected to the peripherals interface 506, such as a global navigationsatellite system (GNSS) (e.g., GPS receiver), a temperature sensor, abiometric sensor, magnetometer or other sensing device, to facilitaterelated functionalities.

A camera subsystem 520 and an optical sensor 522, e.g., a chargedcoupled device (CCD) or a complementary metal-oxide semiconductor (CMOS)optical sensor, can be utilized to facilitate camera functions, such asrecording photographs and video clips. The camera subsystem 520 and theoptical sensor 522 can be used to collect images of a user to be usedduring authentication of a user, e.g., by performing facial recognitionanalysis.

Communication functions can be facilitated through one or more wirelesscommunication subsystems 524, which can include radio frequencyreceivers and transmitters and/or optical (e.g., infrared) receivers andtransmitters. The specific design and implementation of thecommunication subsystem 524 can depend on the communication network(s)over which the computing device 500 is intended to operate. For example,the computing device 500 can include communication subsystems 524designed to operate over a GSM network, a GPRS network, an EDGE network,a Wi-Fi or WiMax network, and a Bluetooth™ network. In particular, thewireless communication subsystems 524 can include hosting protocols suchthat the device 100 can be configured as a base station for otherwireless devices.

An audio subsystem 526 can be coupled to a speaker 528 and a microphone530 to facilitate voice-enabled functions, such as speaker recognition,voice replication, digital recording, and telephony functions. The audiosubsystem 526 can be configured to facilitate processing voice commands,voice printing and voice authentication, for example.

The I/O subsystem 540 can include a touch-surface controller 542 and/orother input controller(s) 544. The touch-surface controller 542 can becoupled to a touch surface 546. The touch surface 546 and touch-surfacecontroller 542 can, for example, detect contact and movement or breakthereof using any of a plurality of touch sensitivity technologies,including but not limited to capacitive, resistive, infrared, andsurface acoustic wave technologies, as well as other proximity sensorarrays or other elements for determining one or more points of contactwith the touch surface 546.

The other input controller(s) 544 can be coupled to other input/controldevices 548, such as one or more buttons, rocker switches, thumb-wheel,infrared port, USB port, and/or a pointer device such as a stylus. Theone or more buttons (not shown) can include an up/down button for volumecontrol of the speaker 528 and/or the microphone 530.

In one implementation, a pressing of the button for a first duration candisengage a lock of the touch surface 546; and a pressing of the buttonfor a second duration that is longer than the first duration can turnpower to the computing device 500 on or off. Pressing the button for athird duration can activate a voice control, or voice command, modulethat enables the user to speak commands into the microphone 530 to causethe device to execute the spoken command. The user can customize afunctionality of one or more of the buttons. The touch surface 546 can,for example, also be used to implement virtual or soft buttons and/or akeyboard.

In some implementations, the computing device 500 can present recordedaudio and/or video files, such as MP3, AAC, and MPEG files. In someimplementations, the computing device 500 can include the functionalityof an MP3 player, such as an iPod™. The computing device 500 can,therefore, include a 36-pin connector that is compatible with the iPod.Other input/output and control devices can also be used.

The memory interface 502 can be coupled to memory 550. The memory 550can include high-speed random access memory and/or non-volatile memory,such as one or more magnetic disk storage devices, one or more opticalstorage devices, and/or flash memory (e.g., NAND, NOR). The memory 550can store an operating system 552, such as Darwin, RTXC, LINUX, UNIX, OSX, WINDOWS, or an embedded operating system such as VxWorks.

The operating system 552 can include instructions for handling basicsystem services and for performing hardware dependent tasks. In someimplementations, the operating system 552 can be a kernel (e.g., UNIXkernel). In some implementations, the operating system 552 can includeinstructions for performing voice authentication. For example, operatingsystem 552 can implement the image processing features as described withreference to FIGS. 1-4.

The memory 550 can also store communication instructions 554 tofacilitate communicating with one or more additional devices, one ormore computers and/or one or more servers. The memory 550 can includegraphical user interface instructions 556 to facilitate graphic userinterface processing; sensor processing instructions 558 to facilitatesensor-related processing and functions; phone instructions 560 tofacilitate phone-related processes and functions; electronic messaginginstructions 562 to facilitate electronic-messaging related processesand functions; web browsing instructions 564 to facilitate webbrowsing-related processes and functions; media processing instructions566 to facilitate media processing-related processes and functions;GNSS/Navigation instructions 568 to facilitate GNSS andnavigation-related processes and instructions; and/or camerainstructions 570 to facilitate camera-related processes and functions.

The memory 550 can store image processing software instructions 572 tofacilitate other processes and functions, such as the image processingprocesses and functions as described with reference to FIGS. 1-4.

The memory 550 can also store other software instructions 574, such asweb video instructions to facilitate web video-related processes andfunctions; and/or web shopping instructions to facilitate webshopping-related processes and functions. In some implementations, themedia processing instructions 566 are divided into audio processinginstructions and video processing instructions to facilitate audioprocessing-related processes and functions and video processing-relatedprocesses and functions, respectively.

Each of the above identified instructions and applications cancorrespond to a set of instructions for performing one or more functionsdescribed above. These instructions need not be implemented as separatesoftware programs, procedures, or modules. The memory 550 can includeadditional instructions or fewer instructions. Furthermore, variousfunctions of the computing device 500 can be implemented in hardwareand/or in software, including in one or more signal processing and/orapplication specific integrated circuits.

We claim:
 1. An image editing method, comprising: rendering adownsampled adjustment image of an input image by determining anadjustment curve for each tile in a plurality of tiles of the inputimage, wherein each adjustment curve defines a relationship of imageediting controls that is applied to pixels in the tile, wherein therelationship is derived from one or more image characteristics of eachtile in the plurality of tiles, and wherein the downsampled adjustmentimage comprises a plurality of pixels and each pixel corresponds to theadjustment curve for a tile in the plurality of tiles; upsampling theadjustment image, by applying an edge preserving upsampling algorithm,to a full size adjustment image that is the same size as the inputimage, wherein each pixel in the full size adjustment image correspondsto a new adjustment curve; and performing pixel by pixel adjustment ofeach pixel in the input image based on the new adjustment curve of acorresponding pixel in the full size adjustment image, thereby obtainingan edited version of the input image.
 2. The image editing method ofclaim 1, further comprising: dividing an input image into the pluralityof tiles, wherein each tile comprises 10 or more pixels in eitherdimension.
 3. The image editing method of claim 1, further comprising:identifying a set of one or more image characteristics for each tile inthe plurality of tiles of the input image.
 4. The image editing methodof claim 1, wherein the adjustment curve defines at least tworelationships of image editing controls that are selected from the groupconsisting of exposure controls, brightness controls, shadows controls,highlights controls, contrast controls, blackpoint controls, andcombinations thereof.
 5. The image editing method of claim 1, whereinthe one or more image characteristics comprise one or more selected fromthe group consisting of a color histogram, a grey histogram, acumulative histogram, statistics, a user input, and combinationsthereof.
 6. The image editing method of claim 1, wherein the adjustmentcurve is selected from the group consisting of a polynomial function, alinear function, a gamma function, an exponential function, a linearequation, a sigmoid function, and combinations thereof.
 7. The imageediting method of claim 1, wherein the adjustment curve is a secondorder polynomial based tone curve: y=ax²+(1−a)x, where x represents aninput data value, y represents an output result, and a is a valuespecific to all pixels in each tile, and wherein x and y correspond topixel values of one or more selected from the group consisting ofexposure, brightness, shadows, highlights, contrast, blackpoint andcombinations thereof.
 8. The image editing method of claim 7, wherein yis set at a predetermined value.
 9. The image editing method of claim 7,wherein y corresponds to the medium brightness level of pixels in a tileof the plurality of tiles.
 10. The image editing method of claim 7,wherein the adjustment curve further defines a relationship of contrastcontrols.
 11. The image editing method of claim 1, wherein a first guideimage and a second guide image are used during the upsampling step. 12.The image editing method of claim 11, wherein the first guide image is agrey version of the input image.
 13. The image editing method of claim12, wherein the second guide image is blurred or downsampled version ofthe first guide image.
 14. A non-transitory machine readable mediumstoring an image-editing application executable by at least oneprocessor, the image-editing application comprising sets of instructionsfor: dividing an input image into a plurality of tiles, wherein eachtile comprises 10 or more pixels in either dimension; rendering adownsampled adjustment image by determining an adjustment curve for eachtile in the plurality of tiles, wherein each adjustment curve defines arelationship of image editing controls that is applied to pixels in thetile, wherein the relationship is derived from one or more imagecharacteristics of each tile in the plurality of tiles, wherein thedownsampled adjustment image comprises a plurality of pixels and eachpixel corresponds to the adjustment curve for a tile in the plurality oftiles; upsampling the adjustment image, by applying an edge preservingupsampling algorithm, to a full size adjustment image that is the samesize as the input image, wherein each pixel in the full size adjustmentimage corresponds to a new adjustment curve; and performing pixel bypixel adjustment of each pixel in the input image based on the newadjustment curve of a corresponding pixel in the full size adjustmentimage.
 15. The non-transitory machine readable medium of claim 14,wherein the image-editing application further comprises sets ofinstructions for: obtaining an edited version of the input image. 16.The non-transitory machine readable medium of claim 14, wherein theimage-editing application further comprises sets of instructions for:identifying a set of one or more image characteristics of each tile inthe plurality of tiles of the input image.
 17. The non-transitorymachine readable medium of claim 14, wherein the adjustment curvedefines at least two relationships of image editing controls that areselected from the group consisting of exposure controls, brightnesscontrols, shadows controls, highlights controls, contrast controls,blackpoint controls, and combinations thereof.
 18. The non-transitorymachine readable medium of claim 14, wherein the one or more imagecharacteristics comprise one or more selected from the group consistingof a color histogram, a grey histogram, a cumulative histogram,statistics, a user input, and combinations thereof.
 19. Thenon-transitory machine readable medium of claim 14, wherein theadjustment curve is selected from the group consisting of a polynomialfunction, a linear function, a gamma function, an exponential function,a linear equation, a sigmoid function, and combinations thereof.
 20. Thenon-transitory machine readable medium of claim 14, wherein theadjustment curve is a second order polynomial based tone curve:y=ax²+(1−a)x, where x represents an input data value, y represents anoutput result, and a is a value specific to all pixels in each tile, andwherein x and y correspond to pixel values of one or more selected fromthe group consisting of exposure, brightness, shadows, highlights,contrast, blackpoint and combinations thereof.
 21. The non-transitorymachine readable medium of claim 20, wherein y is set at a predeterminedvalue.
 22. The non-transitory machine readable medium of claim 20,wherein y corresponds to the medium brightness level of all pixels in atile of the plurality of tiles.
 23. The non-transitory machine readablemedium of claim 20, wherein the adjustment curve further defines arelationship of contrast controls.
 24. The image editing method of claim14, wherein a first guide image and a second guide image are used duringthe upsampling step.
 25. The image editing method of claim 24, whereinthe first guide image is a grey version of the input image.
 26. Theimage editing method of claim 15, wherein the second guide image isblurred or downsampled version of the first guide image.